--- title: Recreating Steffl's Flatfields keywords: fastai sidebar: home_sidebar summary: "Based on his thesis appendix" description: "Based on his thesis appendix" nb_path: "notebooks/06_calib.steffl.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %} {% raw %} 
hv.extension("bokeh", logo=False)
{% endraw %}

missing = [] there = [] for id in obsids: try: data = UVPDS(id, skip_download=True) except FileNotFoundError: print(id, "not there.") missing.append(id) else: print("Got", id) there.append(id)

{% raw %}
{% endraw %} {% raw %} 
cat = CatalogFilter(steffl_spica_dates[2])
{% endraw %} {% raw %} 
pids = list(cat.get_euv_date().query("OBSERVATION_TYPE=='CALIB'").index)
{% endraw %} {% raw %} 
pids
{% endraw %} {% raw %} 
cat.set_next_day()
{% endraw %} {% raw %} 
pids.extend(list(cat.get_euv_date().query("OBSERVATION_TYPE=='CALIB'").index))
{% endraw %} {% raw %} 
pids
{% endraw %} {% raw %} 
kwargs = {"x": "nx", "y": "ny", "cmap": "viridis", "clim": (0, 50)}
{% endraw %}

Row2Row correction

{% raw %}

class Row2Row[source]

Row2Row(pid)

Parameters:

  • pid : <class 'inspect._empty'>
{% endraw %} {% raw %}
{% endraw %} {% raw %} 
pids
{% endraw %} {% raw %} 
r2r = Row2Row(pids[0])
{% endraw %} {% raw %} 
r2r.plot_set
{% endraw %} {% raw %} 
r2r.plot_averaged
{% endraw %} {% raw %} 
r2r.plot_ff
{% endraw %} {% raw %} 
r2r.plot_integrated
{% endraw %} {% raw %} 
r2r.plot_column_std
{% endraw %} {% raw %} 
r2r.ff
{% endraw %}

Col2Col sensitivity variation.

Universal detector stack xarray creator

This function allows to create detector-shaped stacks of data for different purposes, like:

  • along_slit scans
  • across_slit scans
  • sets of corrections for the Steffl flatfield
{% raw %}

create_detector_stack[source]

create_detector_stack(data, name, third_dim, orig)

Function to create a 2D detector array with variable 3rd dimensions.

It uses an orig array to copy the coordinates from.

Parameters:

  • data : <class 'inspect._empty'>

    numpy 3D array to embed to xarray

  • name : <class 'inspect._empty'>

    name of data

  • third_dim : <class 'inspect._empty'>

    name of third dimension ("scan", "along", "across")

  • orig : <class 'inspect._empty'>

    xarray to copy coords from

{% endraw %} {% raw %}
{% endraw %} {% raw %}

class Col2Col[source]

Col2Col(pids, i=15, m=0, plot_width=600) :: Parameterized

params(i=Integer, m=Integer, pids=List, name=String)

Parameters of 'Col2Col'

 Parameters changed from their default values are marked in red. Soft bound values are marked in cyan. C/V= Constant/Variable, RO/RW = ReadOnly/ReadWrite, AN=Allow None

Name Value Type Bounds Mode 

pids [] List (0, None) V RW i 0 Integer (0, 1023) V RW m 0 Integer (0, 13) V RW

Parameter docstrings: =====================

pids: List of product IDs i: Spectral column i m: Scan number m

Parameters:

  • pids : <class 'inspect._empty'>

    group of product ids for a raster run

  • i : <class 'int'>, optional

    Minimum column value for evaluation (15:997)

  • m : <class 'int'>, optional

    Default start scan

  • plot_width : <class 'int'>, optional
{% endraw %} {% raw %}
{% endraw %} {% raw %} 
c2c = Col2Col(pids)
{% endraw %} {% raw %} 
c2c.calculate_simple_correction()
{% endraw %} {% raw %} 
c2c.plot_all_Fm()
{% endraw %}
  • It is neat to see how the grid has a weaker signal in the last array, where only 2 columns could be combined
{% raw %} 
c2c.plot_simple_correction()
{% endraw %}

Row2Row flats

Every across_slit scan comes with an inner along_slit scan that can be used to create a row2row flatfield correction.

So we have 14 of them in this case, that can be averaged.

{% raw %} 
c2c.plot_r2r_flats()
{% endraw %} {% raw %} 
c2c.r2r_flat_average_plot()
{% endraw %}
  • Nice to see how the SNR below 155 nm has improved drastically from having the average of the 14 FF corrections.
{% raw %} 
c2c.simple_both_plot()
{% endraw %} {% raw %} 
c2c.i=100
c2c.m=0
{% endraw %} {% raw %} 
c2c.plot_triplet()
{% endraw %} {% raw %} 
c2c.plot_triplet(corrected=True)
{% endraw %} {% raw %} 
def compare_before_after(i=0):
    p1 = (
        c2c.arr.where(c2c.arr > 0, np.nan)
        .isel(across_slit=i, drop=True)
        .hvplot(
            x="spectral",
            y="spatial",
            cmap="viridis",
            logz=True,
            label="Original",
            # widget_type="scrubber",
            widget_location="bottom",
        )
    )
    p2 = (
        c2c.corrected_arr.where(c2c.corrected_arr > 0, np.nan)
        .isel(across_slit=i, drop=True)
        .hvplot.image(
            x="spectral",
            y="spatial",
            cmap="viridis",
            logz=True,
            label="Corrected",
            # widget_type="scrubber",
            widget_location="bottom",
        )
    )
    return pn.Column(p1) + pn.Column(p2)
{% endraw %} {% raw %} 
slider = pn.widgets.IntSlider(end=14)
{% endraw %} {% raw %} 
pn.interact(compare_before_after, i=(0, 14))
{% endraw %} {% raw %} 
p2 = c2c.corrected_arr.where(c2c.corrected_arr > 0, np.nan).isel(across_slit=i, drop=True).hvplot.image(
    x="spectral",
    y="spatial",
    cmap="viridis",
    logz=True,
    label="Corrected",
    # widget_type="scrubber",
    widget_location="bottom",
)
{% endraw %} {% raw %} 
import panel as pn
{% endraw %} {% raw %} 
pn.Column(p1) + pn.Column(p2)
{% endraw %} {% raw %} 
(p1+p2)
{% endraw %}

Mean of inner area is 1:

{% raw %} 
c2c.simple_both.isel(spectral=slice(15, 998), spatial=slice(3,61)).mean().data
{% endraw %} {% raw %} 
iarr = c2c.arr.interactive()
{% endraw %} {% raw %} 
from panel import widgets
{% endraw %} {% raw %} 
ival = widgets.IntSlider(start=0, end=1024)
mval = widgets.IntSlider()
{% endraw %} {% raw %} 
iarr.isel(spectral=ival, drop=True).hvplot().layout()
{% endraw %} {% raw %} 
c2c.calculate_simple_correction()
{% endraw %} {% raw %} 
c2c.simple_correction
{% endraw %} {% raw %} 
c2c.i = 200
c2c.m = 0
{% endraw %} {% raw %} 
c2c.column_set_mean() / c2c.column_set()[0]
{% endraw %} {% raw %} 
c2c.column_set()[0]
{% endraw %} {% raw %} 
c2c.i = 200
{% endraw %} {% raw %} 
c2c.m = 3
{% endraw %} {% raw %} 
c2c.plot_averaged_triplet()
{% endraw %} {% raw %} 
c2c.plot_triplet(corrected=False, i=300)
{% endraw %} {% raw %} 
c2c.plot_simple_correction()
{% endraw %} {% raw %} 
c2c.simple_correction.shape
{% endraw %} {% raw %} 
c2c.plot_Fm(0)
{% endraw %} {% raw %} 
c2c.plot_triplet()
{% endraw %} {% raw %} 
c2c.column_set_mean()
{% endraw %} {% raw %} 
c2c.plot()
{% endraw %} {% raw %} 
archive_df.loc["EUV2001_093_08_35_28"]
{% endraw %} {% raw %} 
p = obsdir / "index_repaired.tab"
{% endraw %} {% raw %} 
df = pd.read_csv(p, quotechar='"', skipinitialspace=True)
{% endraw %} {% raw %} 
from planetarypy.pds.indexes import find_mixed_type_cols
{% endraw %} {% raw %} 
find_mixed_type_cols(df, fix=False)
{% endraw %} {% raw %} 
df.columns
{% endraw %} {% raw %} 
index.columns
{% endraw %} {% raw %} 
index[index.filename.str.startswith("EUV")].iloc[0]
{% endraw %} {% raw %} 
obs.head()
{% endraw %} {% raw %} 
cols = ["index start_time stop_time detector target obsid_time unknown type comment "]
{% endraw %}